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ABSTRACT 
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test development module that provides instruction on test creation to 
meet an expressed need of faculty; (2) to create a computer-based 
test development tool that facilitates easy test creation; (3) to 
create an experimental tool that helps examine computer-based testing 
processes; and (4) to develop a computer-based module that 
incorporates testing strategies aimed at facilitating higher-order 
learning. TestMaker was designed in HyperCard, and consists of three 
main components--a test creation module, an advisement module, and a 
test module. The Chree modules are examined in detail, 
creating interaction during testing are also reviewed, 
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Introduction 

From the beginning of the 1900s pape.--and-pencil testing has been the standard means of administering 
achievement tests and many other educational measurements to learners (Olsen, Maynes, Slawson, & 
Ho, 1989). The proliferation of computer technology in society within the last forty years and the 
decreasing cost of microcomputers have enabled computers to be utilized for a variety of educational 
purposes. Some of the more standard uses have been in the area of assessment (Eaves & Smith, 1986). 

Computers serve numerous functions related to assessment. Advancements in technologies enhance the 
computer's potential for improving lest administration, scoring and reporting (Olsen, Maynes, Slawson, 
& Ho, 1989). Computers are capable of managing many of the clerical duties embedded in the 
assessment process. Moreover, they broaden our expectations and provide worthwhile insights into 
what should be measured and the methods most conducive for measurement (Johnson & Hariow, 1989). 
They provide educators with a more accurate experimental tool with which to manage the delivery of 
instruction, as well as learner assessment. Computers are not only capable of presenting test items but 
evaluating them as well. 

Several factors may be attributed to the assimilation of computers into educational assessment, some of 
which are presented below. With conventional paper-and-pencil tests there is a considerable time lapse 
between the time of testing and the time at which test results are reported. These shortcomings prevent 
them from having much use in assisting teachers and learners in everyday educational pursuits. 
Consequently, educators and instructional designers have directed their efforts to finding new 
techniques and tools in order to curtail the current limitations associated with many assessment 
methods. Their attention, in many cases, has been focused on computer technology and, as Johnson and 
Hariow (1989) point out, computers are the likely tools for enriching the theory and practice of 
educational assessment. 

A second factor influencing trends toward using computers in educational assessment is the 
administrative advantage this technology offers. Eaves and Smith (1986), for example, support 
computer use for learner assessment because of increased efficiency in test duplication, administration, 
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scoring, item banking, record keeping and analysis. The major benefit computers provide relative to 
printed test administration are evident in two primary areas: test administration and test evaluation. 
Some enhancements related to test administration are standardized test administration conditions, 
individually administered tests, immediate test scoring, increased variety of testing formats, and ability 
to collect test and item latency information (Olsen, Maynes. Slawson. and Ho. 1989). Johnson and 
Harlow (1989) point out that the primary advantages that computers offer evaluation are effective and 
efficient scoring, objectivity - computers have the advantage of being totally objective which is not 
possible with a human, and more organized assessment. 

Assessment is a fundamental part of education. Traditionally, multiple-choice, true/false and object type 
tests have been the predominant means of assessment. While used heavily and appropriate in many 
cases, current educational perspectives (e.g.. constructivism) require that assessment also focuses on 
higher order thinking and problem-solving abilities. Operations and strategies which are incorporated 
in courseware must account for and facilitate deeper levels of cognitive processing (Jonassen. 1988). A 
potential advantage of computerized testing is the capability to allow users to interact with testing. 
Research suggests that allowing for interaction during testing promotes learning. For example, giving 
learners the opportunity to respond to feedback was found to facilitated higher levels of achievement. 
Activities during computerized testing which help learners relate new information to existing 
knowledge (e.g.. responding to feedback) may prove beneficial to learning and should be considered by 
instructional designers. It is possible that as computerized testing procedures become more interactive 
and dynamic, higher order learning may be facilitated. 

The research literature appears to support the claim that computer and papcr-and-pcncil test 
administration produce comparable results in learner achievement scores. Studies investigating this 
question generally indicate computer testing to be closely equivalent to paper-and-pencil tests in terms 
of achievement scores. Stile and Pcttibone (1983), for example, found no significant differences in 
acquired scores when comparing computer testing to paper-and-pencil. Eaves and Smith (1986) report 
that undergraduate college students' achievement scores on computer administered tests are similar to 
that of papcr-and-pcncil. Olscn. Maynes, Slawson. and Ho (1989) show that the scores obtained from 
paper-administcrcd tests, computer-administered tests, and computer-adaptive tests are equivalent. 

The paper will discuss a computer-base prototype called TestMakcr that enables educators to create 
computer-based tests. The prototype is being designed to incorporate student record keeping and 
scoring, techniques for interactive testing, and advisement on test creation. Strategies for creating 
interaction during testing are also reviewed. 

TestMaker Objectives 

Despite the fact that many programs that have been created for computer-based test construction, 
educators frequently rely on paper and pencil. Several reasons may account for this such as the 
unavailability of technology, unfarniliarity vAih technology and/or the inability to tailor test creation 
programs to meets individual needs. The development of TestMaker was initiated by several requests 
from faculty at Eastern Illinois University for a computer-based test development tool that . among 
other things, would: 1) provide instruction on effective strategics for creating tests and; 2) assist in the 
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creation of computerized tests that evaluate and record student performance. The program is designed 
as an instructional and developmental tool for teacher-education students. 

Given the functional needs of faculty, the host of research implications computer technology has for 
assessment, and current educational perspectives such as the constructivism and their impact on testing; 
the purposes for developing TestMaker were: 

1) 10 create a test development module that provides instruction on test creation to meet 
an expressed need of faculty ; 

2) to create a computer-based test development tool that facilitates easy test creation to 
meet an expressed need of faculty, 

3) to create an experimental tool that helps examine computer-based testing processes and; 

4) to develop a computer-based module that incorporates testing strategies aimed at 
facilitating higher order learning. 

Overview of Prototype Design 

TestMaker is presently a prototype that consists of three main components: a test creation module, an 
advisement module and a test module. The test creation module enables developers to select a test type 
(e.g., multiple-choice). Once a test type is selected the developer can enter questions, feedback, and 
subsequently create a student test file. With the current pro{otypc, multiple-choice tests can be created. 
It will eventually include true/false, matching and open-ended response test types. Running 
concurrently with the test creation module is advisement . At ariy point during test creation developers 
have the option to execute the advisement module to obtain information about such topics as writing 
effective questions and t>pes of feedback. The third component is the actual test. From the test 
creation module a student test file is created. This module is fully executable and thus a student sitting 
at a computer can take the test, interact with it and receive feedback on their performance. 

The prototype was designed in HyperCard. The designers pilot tested the instrument to determine the 
type of design features potential users needed and wanted. After some initials trial tests it became 
apparent that the test development module would require greater flexible than could be provided with 
HyperCard. While HyperCard proved useful for getting a model up and running, the designers decided 
that when full development is implemented a programming language (e.g., C) will be used. 

Modules of TestMaker 

Advisement Module 

Teacher-education students arc the primary audience for which this program was designed. Many of 
these students have limited knowledge of effective test creation. Thus, the designer needed to create a 
facility in which developcn; could obtain advice on how to effectively create multiple-choice, true false, 
matching and open-ended questions. Two types of advisement are provided: user selected advice and 
system generated advice. TestMaker includes an advisement module which runs concurrently with the 
test creation program so that users can select key words to access advice on particular topics. For 
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example, developers can get advice on types of feedback, hints for writing test items, test items length 
and complexity (see Figure 1). This advisement is referred to as user-selected advice. When 
developing a test, the prototype will automatically provide adyjce on effective testing strategies (e.g., 
suggesting an appropriate number of alternatives for multiple choice questions if the user attempts to 
enter too many). This advisement is referred to as system generated advice. 

Figure 1 

Screen Example: Advisement Module with TestMaker 
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Test Creation Module 

This prototype is being designed as a test development tool and an instructional module. To create a 
multiple choice test, for example, the user enters each question^ a correct answer and ahematives simply 
by typing in predefined text blocks (see Figure 2). Previously created questions can be viewed and 
edited. Prompts guide users through the test creation process (e.g., "Enter the number of alternatives." 
or "Identify the correct answer.") and when necessary the system generates advice. For instance, if a 
user attempts to create 15 alternatives for a particular test item, the system indicates that too many 
alternatives are requested and it recommends an appropriate number to use for that test item. 
User-selected advice is available at any time during test item creation. 
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Figure 2 

Screen Example: Creation Module 




For each question, developers can incoiporate feedback which will be displayed for students during 
testing. As a form of feedback, the correct answer can be presented for incorrect responses. Thus, if 
the instructor enters feedback and selects the correct answer display option, then; upon responding 
incorrectly to a test item, the student would be presented \he correct answer along with the instructor's 
feedback. 

Also available during development is an option for interaction during testing. While still being refined, 
this feature allows test takers to respond to feedback, explain their answers and to defend why they are 
right or wrong. For example, if the instructor had selected this option, then; upon responding to a 
particular test item, students would be prompted to elaborate as to why their response was correct. The 
student would then type an explanation which could later be reviewed by the instructor. Requesting 
students to elaborate on their responses is intended to facilitate the integration of new information or not 
yet learned content with their existing knowledge. Elaboration includes meaningful additions or 
constructions which help to improve an individual's recollection of learned content (Levin, 1988). 
Elaboration is hypothesized to facilitate infomiation recall by generating alternate information retrieval 
.channels and by providing additional information for the formulation of responses (Gagn6, 1985). 
Marrone (1990) found that allowing students to respond to feedback in computerized testing facilitated 
higher levels of achievement. In this respect, the testing experience becomes itself a type of learning 
strategy which helps to facilitate learning. 

Testing Module 

From the test creation module the actual test file is created. This file can be given to students on disk or 
over a local area networks. Students, taking tlic test arc presented questions to which they respond. All 
question items, correct answers, alternatives and feedback created in the lest creation module arc 
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presented. Students are given a grade upon completion. During testing, student responses, elaborations 
and instructional time are recorded to a file which can later be reviewed by the instructor. 

Summary 

Computer-based testing has existed for several years. It appears to offer many advantages to educators 
including effective and efficient grade reporting and immediate feedback to students. A primary 
function of the testing program discussed in this paper is to help individuals learn how to construct 
effective tests and to provide them a means by which to do so. Tlius, by creating an easy-to-use test 
development tool, which incorporates system-generated and user-selected advisement, it is hoped that 
the intended audience will be better able to create effective assessment instruments. Moreover, when 
considering assessment, in light of constructivist approaches, it is necessary to view measurement of 
learning in terms of higher order processing and problems solving. This module, while intended to 
meet faculty needs for objective tests (e.g., multiple-choice), includes features that will potentially help 
students generate their own meaning from information and subsequently construct knowledge. Features 
such as interactive testing and allowing learners to pose arguments as to why responses are right or 
wrong may be facilitative of higher order learning and are deserving of further research. 

References: 

Eaves, R. C. & Smith, E. (1986). The effect of media and amount of microcomputer 
experience on examination scores. Journal of Experiential Education, 55(1), 23-36. 

Gagn6, E. (1985). The cognitive psychology of school learning. Boston: Little, Brown and 
Company. 

Johnson, D. L. & Harlow, S. D. (1989). Using the computer in educational assessment: Can 
it be of any help? Diagnostique, 14(3), 274-28 1 . 

Jonassen, D.H. (1988). Instructional designs for microcomputer courseware. Hillsdale, New 
Jersey: Lawrence Erlbaum Associates. 

Levin, J. R. (1988). Elaboration-based learning strategies powerful theory=powerful 
application. Contemporary Educational Psych, 13, 191-205. 

Marrone, J. A. (1990). Learner response to computerized answer judging and its effect on 
learning and retention. Doctoral dissertation. The Pennsylvania State University. 

Olsen, J. B., Maynes, D. D., Slawson, D. & Ho, K. (1989). Comparisons of 
paper-administered, computer administer and computer-adaptive achievement test. Journal 
of Computing Research, 5(3), 31 1-326. 

Stile, S. & Pettibone, T. J. (1983). Micro-computer assisted student examination in special 
education training. Teacher Education and Special Education, 6, 192-195. 



73 



